Transparent and Flexible Network Management for Big Data Processing in the Cloud
نویسندگان
چکیده
We introduce FlowComb, a network management framework that helps Big Data processing applications, such as Hadoop, achieve high utilization and low data processing times. FlowComb predicts application network transfers, sometimes before they start, by using software agents installed on application servers and while remaining completely transparent to the application. A centralized decision engine collects data movement information from agents and schedules upcoming flows on paths such that the network does not become congested. Results on our lab testbed show that FlowComb is able to reduce the time to sort 10GB of randomly generated data by 35% while changing paths for only 6% of the transfers.
منابع مشابه
Cloud Computing Technology Algorithms Capabilities in Managing and Processing Big Data in Business Organizations: MapReduce, Hadoop, Parallel Programming
The objective of this study is to verify the importance of the capabilities of cloud computing services in managing and analyzing big data in business organizations because the rapid development in the use of information technology in general and network technology in particular, has led to the trend of many organizations to make their applications available for use via electronic platforms hos...
متن کاملApplication of Big Data Analytics in Power Distribution Network
Smart grid enhances optimization in generation, distribution and consumption of the electricity by integrating information and communication technologies into the grid. Today, utilities are moving towards smart grid applications, most common one being deployment of smart meters in advanced metering infrastructure, and the first technical challenge they face is the huge volume of data generated ...
متن کاملA Model based on Cloud Computing for the implementation and management IT services in Banks
In recent years, the banking industry has made significant changes in technology and communications. The expansion of electronic communications and a large number of people around the world access to the Internet, appropriate to establish trade and economic exchanges provided but high costs, lack of flexibility and agility in existing systems because of the large volume of information, confiden...
متن کاملPrivacy and Security of Big Data in THE Cloud
Big data has been arising a growing interest in both scien- tific and industrial fields for its potential value. However, before employing big data technology into massive appli- cations, a basic but also principle topic should be investigated: security and privacy. One of the biggest concerns of big data is privacy. However, the study on big data privacy is still at a very early stage. Many or...
متن کاملA Model based on Cloud Computing for the implementation and management IT services in Banks
In recent years, the banking industry has made significant changes in technology and communications. The expansion of electronic communications and a large number of people around the world access to the Internet, appropriate to establish trade and economic exchanges provided but high costs, lack of flexibility and agility in existing systems because of the large volume of information, confiden...
متن کامل